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I. REAL PARTY IN INTEREST 

The subject patent application was assigned by inventor Irene V. Hu to Fernandez & 
Associates, LLP on June 8, 2004. This assignment was recorded on August 3, 2004 at 
Reel/Frame 015649/0423. A corrective assignment by inventor Irene V. Hu to co-inventor 
Dennis S. Fernandez was executed on June 8, 2004. This assignment was recorded on August 8, 
2005 at Reel/Frame 016343/0986. The subject patent application was assigned by Dennis S. 
Fernandez to Lot 3 Acquisition Foundation, LLC on May 16, 2007. This assignment was 
recorded on July 3, 2007 at Reel/Frame 019515/0553. Therefore, Lot 3 Acquisition Foundation, 
LLC is the real party in interest in the subject patent application. 
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II. RELATED APPEALS AND INTERFERENCES 

Prior to acquisition of the present application by Lot 3 Acquisition Foundation, LLC, a 
Notice of Appeal was filed August 24, 2006, and an amended Appeal Brief was filed November 
17, 2006 in the present application. However, the Examiner reopened prosecution in an Office 
Action dated March 12, 2007, so the appeal was not instituted. Furthermore, a Notice of Appeal 
was filed on February 8, 2010 and an Appeal Brief was filed April 2, 2010 in the present 
application. However, the Examiner reopened prosecution in an Office Action dated July 14, 
20 10. 1 Thus, there are no related appeals and interferences. 



' In the Office Action, the Examiner states that the reopening of prosecution was due to a Board decision, 
but the Examiner's statement is incorrect — there has been no Board decision in this application. 
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III. STATUS OF CLAIMS 

Claims 24-33 and 39-53 are pending in the application, of which claims 24 and 3 1 are 
independent claims. Claims 24-33 and 39-53 are rejected under 35 U.S.C. § 103(a). Claims 1- 
23 and 34-38 were previously cancelled. All claims 24-33 and 39-53 are being appealed. 
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IV. STATUS OF AMENDMENTS 

No amendments were presented in reply to the Final Office Action dated November 19, 
2009 or in reply to the Office Actions dated July 14, 2010 and August 30, 2010. Thus, the 
claims in the Claims Appendix represent the claims Appellant believes to be currently pending. 
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V. SUMMARY OF CLAIMED SUBJECT MATTER 

Independent claim 24 is directed to a system that comprises "a communicator" 2 that 
receives "first data" 3 and "second data" 4 associated with an "object." 5 The first data is received 
from a "fixed detector," 6 and the second data is received from a "mobile target unit." 7 The 
mobile target unit comprises a sensor that detects the second data. 8 The mobile target unit "is at 
least one of: mounted in the object, mounted on the object, carried in the object, or carried on the 
object." 9 The system further comprises "a processor" 10 that correlates the first data and the 
second data to generate "object location information." 11 

Independent claim 3 1 12 is directed to a method that comprises receiving first data from a 
fixed detector and second data from a mobile target unit. 13 The first data and second data are 



2 See, e.g., p. 3, line 7 through p. 6, line 5; p. 8, lines 22-30; p. 12, lines 8-12; and FIGS. 1, 3. Citations 
are to page and line numbers from the application as filed for Serial No. 09/823,509. 

3 See, e.g., p. 7, line 1 through p. 8, line 4; p. 8, lines 22-30; p. 9, lines 9-18; p. 10, line 7 through p. 1 1 
line 27; p. 18, line 17 through p. 19, line 19; p. 29, line 17 through p. 32, line 26; and p. 33, line 27 
through p. 35, line 12. 

4 See, e.g., p. 7, line 1 through p. 8, line 4; p. 8, line 22 through p. 9, line 18; p. 10, line 7 through p. 1 1 
line 27; p. 18, line 17 through p. 19, line 19; p. 29, line 17 through p. 32, line 26; and p. 33, line 27 
through p. 35, line 12. 

5 See, e.g., p. 2, lines 3-15; p. 3, lines 12-19; p. 11, line 18 through p. 12, line 6; p. 17, lines 1-8; p. 19, 
lines 21-29; p. 23, line 25 through p. 25, line 28; and p. 26, line 24 through p. 27, line 17. 

6 See, e.g., p. 5, line 27 through p. 8, line 30; p. 27, lines 1-17; p. 28, lines 4-15; p. 30, line 4 through p. 
31, line 10; and p. 34, lines 24-29. 

7 See, e.g., p. 5, line 27 through p. 6, line 29; p. 9, lines 1-5; p. 27, lines 1-17; p. 35, lines 1-8; and FIGS. 
1-2. 

8 See, e.g., p. 5, line 27 through p. 6, line 29; p. 9, lines 1-5; p. 27, lines 1-17; p. 35, lines 1-8; and FIGS. 
1-2. 

9 See, e.g., p. 9, lines 1-18; p. 1 1, line 29 through p. 12, line 12; and p. 13, lines 9-16. 

10 See, e.g., p. 3, line 2 through p. 6, line 5; p. 9, lines 20-27; p. 10, line 17 through p. 1 1, line 20; p. 13, 
line 24 through p. 14, line 22; and FIGS. 2-3. 

" See, e.g., p. 10, line 17 through p. 1 1, line 20; p. 18 line 17 through p. 21, line 20; p. 22, lines 7-13; p. 

29, line 17 through p. 32, line 26; p. 33, line 1 through p. 35, line 8; and FIG. 4. 

12 See, e.g., FIG. 4 and support for similar elements shown with respect to claim 24 listed above. 

,3 See, e.g., p. 2, lines 3-15; p. 3, lines 12-19; p. 7, line 1 through p. 8, line 4; p. 8, lines 22-30; p. 9, lines 

9-18; p. 10, line 7 through p. 11 line 27; p. 17, lines 1-8; p. 18, line 17 through p. 19, line 19; p. 23, line 

25 through p. 25, line 28; and p. 26, line 24 through p. 27, line 17; p. 29, line 17 through p. 32, line 26; 

and p. 33, line 27 through p. 35, line 12. 
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associated with an object. 14 The mobile target unit is mounted in, mounted on, and/or carried on 
the object. 15 The first data and the second data are correlated to generate object location 
information. 16 

No means plus function or step plus function claims under 35 U.S.C. § 1 12, sixth 
paragraph are being appealed. 



14 See, e.g., p. 2, lines 3-15; p. 3, lines 12-19; p. 1 1, line 18 through p. 12, line 6; p. 17, lines 1-8; p. 19, 
lines 21-29; p. 23, line 25 through p. 25, line 28; and p. 26, line 24 through p. 27, line 17. 

15 See, e.g., p. 9, lines 1-18; p. 1 1, line 29 through p. 12, line 12; and p. 13, lines 9-16. 

16 See, e.g., p. 10, line 17 through p. 11, line 20; p. 18 line 17 through p. 21, line 20; p. 22, lines 7-13; p. 
29, line 17 through p. 32, line 26; p. 33, line 1 through p. 35, line 8; and FIG. 4. 



VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Whether claims 24-33 and 39-53 are unpatentable under 35 U.S.C. § 103(a) over 
Moengen, U.S. Patent No. 6,373,508 ("Moengen"). 
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VII. ARGUMENT 

Appellant respectfully requests that the Board reverse the Examiner's rejections under 35 
U.S.C. § 103(a) over Moengen. 

A. Claims 24-26, 28-33, 39-40, and 42-53 

Moengen discloses "a method for manipulation of a movable object displayed in a 
television picture, [where] the distance between the object and fixed basic positions is detected at 
a time t together with the distance between the object and a television camera in a known 
position" (Abstract; emphasis added). "Both the position detectors [D] and the television 
cameras [K] are positioned in a pre selected x,y,z coordinate system . . . The positions of both 
the position detectors [D] and the cameras [K] are precisely defined in the x,y,z co-ordinate 
system" (Moengen, column 5, lines 51-59; emphasis added). 

"Where mobile cameras ... are used, it will be possible to determine the camera 
positions by means of the position detectors [D]" (column 12, lines 28-31; emphasis added). 
"[T]he position of a mobile camera ... is determined by means of GPS and transferred to the 
production location" (column 16, lines 17-20). "Four position detectors [D] are preferably used 
to achieve unambiguous detection of the position of the natural object N. The position of the 
object N is thereby solely determined by distance measurements, i.e. by trilateration" (column 6, 
lines 39-43; emphasis added). "The detected distances are given to a computing module 2 which 
by means of trilateration calculates the positions x,y,z at different times t and thereby also the 
path of the object N on the basis of positions detected at the different times t" (column 7, lines 4- 
8; emphasis added). 

Because "the position of [Moengen's] object N is . . . solely determined by distance 
measurements [of fixed position detectors D]" (column 6, lines 39-43; emphasis added) (see also 
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column 5, lines 51-59), Moengen does not disclose or contemplate, alone or in combination with 
any of the cited references, "a processor configured to correlate the first data [from the fixed 
detector] and the second data [from the mobile target unit] to generate object location 
information" as recited in claim 24 (emphasis added), and as similarly recited in claim 3 1 . 

The Examiner asserts "the camera detects the second data" (Aug. 30, 2010 Office Action, 
page 3). Regardless of whether or not Moengen's camera receives "second data," it is clear from 
Moengen that nothing from Moengen's camera is used to "generate object location information" 
(Appellant's claim 24). For example, Moengen explicitly states, "the position of [Moengen's] 
object N is . . . solely determined by distance measurements" (Moengen, column 6, lines 39-43; 
emphasis added), which explicitly excludes using anything other than "distance measurements" 
to determine position. Therefore, Moengen does not disclose or contemplate, "a processor 
configured to correlate the first data and the second data to generate object location 
information" as recited in claim 24 (emphasis added), and as similarly recited in claim 3 1 . 

Furthermore, Moengen teaches away from Appellant's claims . To clarify, in the 
situation in Moengen where "the natural object is located beyond the range of the position 
detectors ... the position of the natural object must be determined by other means [that is, not 
by the position detectors]" (column 16, lines 11-14; emphasis added). "In this case, the natural 
object may be equipped with a GPS (Global Positioning System) receiver for determination of 
the position" (column 16, lines 14-16). Thus, Moengen discloses use of only the "position 
detectors" or a "GPS receiver" and not both . For this additional reason, Moengen does not 
disclose or contemplate, and in fact teaches away from , "a processor configured to correlate the 
first data and the second data to generate object location information" as recited in claim 24 
(emphasis added), and as similarly recited in claim 3 1 . No "correlation" to "generate object 
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location information" is disclosed or contemplated in Moengen, because Moengen explicitly 
states that either a "position detector" or a "GPS receiver" is used, and not both . 

Moreover, in the case where "the natural object is located beyond the range of the 
position detectors" (Moengen column 16, line 12), there is no "fixed detector" (claims 24 and 
31). Thus, Moengen does not disclose or contemplate, and in fact teaches against both a "fixed 
detector" and a "mobile target unit" where "a processor [is] configured to correlate the first data 
[from the fixed detector] and the second data [from the mobile target unit] to generate object 
location information" as recited in claim 24, and as similarly recited in claim 3 1 . Appellant 
therefore respectfully requests that the Board reverse the Examiner's rejections of claims 24 and 
31. 

The Examiner argues that the "natural object" is represented "in a natural television 
picture in such a manner that the object's position and movement are clearly visible in the 
television" (Aug. 30, 2010 Office Action, page 7). But the Examiner acknowledges that this 
representation of the "natural object" is just that — a representation: "[the] synthetic object 
represents the] position of the natural object" (Aug. 30, 2010 Office Action, page 7). The 
"synthetic object" is not the "natural object": "Even though the natural object may be hidden by 
obstacles which preventfs] it from being viewed directly in the television picture, its position 
and movement can still be indicated by a synthetic object or a synthetic track which is inserted in 
the television picture" (Moengen, column 16, lines 29-33; emphasis added). Thus, any 
representation of the synthetic object in the television picture is not utilized "to generate object 
location information" (claim 24), because the image of the synthetic object is not the natural 
object. 
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In fact, Moengen states that "the positioning accuracy [of the synthetic object] is not 
particularly critical and may well amount to several meters, or possibly several tens of meters" 
(column 16, lines 34-36; emphasis added). Thus, it is nonsensical for the Examiner to argue that 
the synthetic image is actually viewed by the camera, because "the natural object may be hidden 
. . . which prevents] it from begin viewed directly" (column 16, lines 30-31). The "synthetic 
object" represented in the television is clearly not Moengen' s natural object. The synthetic 
object therefore is not "first data . . . received from a fixed detector ... or second data . . . 
received from a mobile target unit" (claim 24). 

In any case, regardless of the arguments the Examiner presents, Moengen clearly states, 
the "position of the object N is thereby solely determined by distance measurements, i.e. by 
trilateration" (column 6, lines 39-43; emphasis added). Thus, Moengen explicitly teaches 
against "correlating the first data [received from a fixed detector] and the second data [received 
from a mobile target unit] to generate object location information" as recited in claim 31, and as 
similarly recited in claim 24. Appellant therefore respectfully requests withdrawal of the 
rejections of claims 24 and 31. 

Claims 25-26, 28-30, 32-33, 39-40, and 42-53 variously depend from independent claims 
24 and 31. Therefore, Appellant asserts that dependent claims 25-26, 28-30, 32-33, 39-40, and 
42-53 are differentiated from the cited references for at least the same reasons stated above for 
differentiating independent claims 24 and 3 1 , as well as in view of their own respective features. 
Appellant thus requests that the Board reverse the Examiner's rejection of claims 25-26, 28-30, 
32-33, 39-40, and 42-53. 
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B. Claim 27 

As discussed above, in the situation in Moengen where "the natural object is located 
beyond the range of the position detectors ... the position of the natural object must be 
determined by other means [that is, not by the position detectors]" (column 16, lines 11-14; 
emphasis added). "In this case, the natural object may be equipped with a GPS (Global 
Positioning System) receiver for determination of the position" (column 16, lines 14-16). Thus, 
Moengen discloses use of only the "position detectors" or a "GPS receiver" and not both . For 
this additional reason, Moengen does not disclose or contemplate, and in fact teaches away 
from , "a processor configured to correlate the first data and the second data to generate object 
location information" as recited in claim 24 (emphasis added), and as similarly recited in claim 
31. No "correlation" to "generate object location information" is disclosed or contemplated in 
Moengen, because Moengen explicitly states that either a "position detector" or a "GPS receiver" 
is used, and not both . 

Furthermore, in the case where "the natural object is located beyond the range of the 
position detectors" (Moengen column 16, line 12), there is no "fixed detector" (claims 24 and 
31). Thus, Moengen does not disclose or contemplate, and in fact teaches against both a "fixed 
detector" and a "mobile target unit" where "a processor [is] configured to correlate the first data 
[from the fixed detector] and the second data [from the mobile target unit] to generate object 
location information" as recited in claim 24, and as similarly recited in claim 3 1 . Therefore, 
Moengen also does not disclose or contemplate "wherein the mobile target unit is at least one of: 
mounted in the object, mounted on the object, carried in the object, or carried on the object" 
(claim 27), because Moengen does not disclose or contemplate the claimed "mobile target unit." 
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Appellant therefore respectfully requests that the Board reverse the Examiner's rejection of claim 
27. 

C. Claim 41 

Moengen discloses an "object vector" and a "camera vector" (see Moengen, column 5, 
line 67 through column 6, line 1 1), but neither of these vectors is a "movement vector" as recited 
in Appellant's claim 41 . For example, the "picture axis or the optical axis in a camera K is 
represented by a vector called the camera vector" (column 5, line 67 through column 6, line 2), 
and the "connecting line between the lens centre in a camera K and the object N ... is called the 
object vector" (Moengen, column 6, lines 12-14). These vectors are used to determine a current 
position of the natural object 17 so that a "synthetic track can be generated in a television picture, 
the synthetic track thus being intended to represent the path of a natural object N in the television 
picture during a given period" (column 14, lines 12-16). Thus these vectors are not "movement 
vectors," but are simply vectors that relate a current position of the natural object to a vector 
associated with a camera. 

Furthermore, the data from the camera vectors and object vectors are used to generate 
this synthetic path that the natural object has already traveled: "[t]he primary object here is that 
the synthetic object S which represents the natural object N should at all times display the 
position and/or the movement of the natural object N, as it would be represented in the 
television picture at any time F (Moengen, column 7, lines 43-47; emphasis added). Therefore, 
Moengen does not disclose or contemplate "determining a movement vector to predict a future 



17 "The data processing unit Q also comprises a manipulator module 4 for generating a synthetic object S 
which corresponds to the natural object N . . . Thus in the position X, Y in the camera's picture plane, via, 
e.g., a video generator ... a synthetic object can be created, representing the natural object N in its 
position X, Yin the picture plane in the camera K" (Moengen, column 7, lines 31-41; emphasis added). 
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location of the object" as recited in claim 41 (emphasis added). Appellant therefore respectfully 
requests that the Board reverse the Examiner's rejection of claim 41 . 
D. Conclusion 

In conclusion, Appellant respectfully requests that the Board reverse the Examiner's 35 
U.S.C. § 103(a) rejections of all pending claims 24-33 and 39-53. 

If it is determined that there are any additional fees associated with this Appeal Brief, the 
United States Patent and Trademark Office is requested to consider this Appeal Brief (with a 
petition if necessary) and charge the appropriate fee to Deposit Account No. 19-2814. This does 
not authorize payment of the issue fee. 

Respectfully submitted, 

David G. Barker 
Reg. No. 58,581 



SNELL & WILMER L.L.P. 

400 E. Van Buren 
One Arizona Center 
Phoenix, Arizona 85004 
Phone: 602-382-6376 
Fax: 602-382-6070 
Email: dbarker@swlaw.com 
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VIII. CLAIMS APPENDIX 

1-23. (Canceled) 

24. (Rejected) A system comprising: 

a communicator configured to receive first data associated with an object and second data 
associated with the object, wherein the first data is received from a fixed detector configured to 
detect the first data, and wherein the second data is received from a mobile target unit 
comprising a sensor configured to detect the second data, wherein the mobile target unit is at 
least one of: mounted in the object, mounted on the object, carried in the object, or carried on the 
object; and 

a processor configured to correlate the first data and the second data to generate object 
location information. 

25. (Rejected) The system of Claim 24 wherein the communicator is further configured to 
receive a target unit location, the processor being further configured to determine whether the 
mobile target unit is within a range of the fixed detector. 

26. (Rejected) The system of Claim 24 wherein: 

the object location information comprises at least one of object trajectory information, 
object physical location information, or object speed information; and 

the fixed detector is configured to provide an image of the object. 

27. (Rejected) The system of Claim 24 wherein the object is a vehicle. 

28. (Rejected) The system of Claim 24, further comprising a database configured to maintain 
a plurality of current positions associated with at least one of a plurality of sensors, a plurality of 
mobile target units, or a plurality of objects. 
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29. (Rejected) The system of Claim 24 wherein the mobile target unit comprises an 
accelerometer configured to provide data indicative of movement of the object to facilitate 
generating the object location information. 

30. (Rejected) The system of Claim 24 wherein: 
the object is an identified good; 

the mobile target unit comprises a radio-frequency identification device; and 

the fixed detector comprises a camera for observing the identified good, to facilitate 

enabling the sensor and the fixed detector to provide corroborative surveillance of the identified 

good. 

3 1 . (Rejected) A method comprising: 

receiving, from a fixed detector, first data associated with an object; 

receiving, from a mobile target unit, second data associated with the object, wherein the 
mobile target unit comprises a sensor configured to detect the second data, and wherein the 
mobile target unit is at least one of: mounted in the object, mounted on the object, carried in the 
object, or carried on the object; and 

correlating the first data and the second data to generate object location information. 

32. (Rejected) The method of Claim 3 1 , further comprising activating a second fixed detector 
in response to the object location information. 

33. (Rejected) The method of Claim 3 1 wherein the second data comprises an object 
identifier, the method further comprising registering the object identifier in a database to indicate 
association with the object. 

34-38. (Canceled). 
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39. (Rejected) The system of Claim 24 wherein the second data comprises the target unit 
location. 

40. (Rejected) The method of Claim 3 1 , wherein the correlating the first data and the second 
data comprises determining compliance with a scheduled object activity. 

4 1 . (Rejected) The method of Claim 3 1 , wherein the correlating the first data and the second 
data comprises determining a movement vector to predict a future location of the object. 

42. (Rejected) The system of Claim 24 further comprising a plurality of detectors each 
having a corresponding observation range, wherein at least one of the plurality of detectors is 
selected to observe the object. 

43. (Rejected) The system of Claim 24 wherein the first data comprises at least one of an 
image of the object or an identifier associated with the object. 

44. (Rejected) The system of Claim 24 wherein the first data comprises a plurality of images 
of the object captured at different times. 

45. (Rejected) The system of Claim 24 wherein the second data comprises at least one of an 
image of the object or an identifier associated with the object. 

46. (Rejected) The system of Claim 24 wherein the second data comprises a plurality of 
images of the object captured at different times. 

47. (Rejected) The system of Claim 24 wherein the object location information is determined 
at least in part based on a fixed detector location. 

48. (Rejected) The system of Claim 24 wherein the object location information is determined 
at least in part based on a mobile target unit location. 

49. (Rejected) The system of Claim 24, further comprising a movement module configured 
to activate a second fixed detector in response to the object location information. 
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50. (Rejected) The method of Claim 31, wherein correlating the first data and the second data 
to generate object location information comprises determining at least one of a trajectory or a 
speed of the object. 

5 1 . (Rejected) The system of Claim 24, wherein the mobile target unit comprises a locator 
unit configured to determine the target unit location. 

52. (Rejected) The system of Claim 24, wherein the fixed detector is configured to be 
selected in response to the processor's correlation of the first data and the second data. 

53. (Rejected) The system of Claim 49, wherein the fixed detector is further from the second 
fixed detector than from a third fixed detector. 



20 



IX. EVIDENCE APPENDIX 

Copies of: 

Moengen, U.S. Patent No. 6,373,508 
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Fig. 3c 



Fig. 3d 
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METHOD AND SYSTEM FOR 
MANIPULATION OF OBJECTS IN A 
TELEVISION PICTURE 

BACKGROUND OF THE INVENTION 

The invention concerns a method for manipulation of at 
least one movable, natural object in a natural television 
picture, wherein the television picture is generated by one or 
more television cameras. The invention also concerns a 
method for generating at least one synthetic track in a 
television picture, wherein the synthetic track represents the 
path of a movable natural object in a natural television 
picture during a given period 6, and wherein there is 
employed a method for manipulation of the movable, natural 
object in the television picture. Finally, the invention con- 
cerns a system for implementation of the method for 
manipulating at least one movable, natural object in a natural 
television picture, wherein the television picture is generated 
by one or more television cameras, together with implemen- 
tation of the method for generating a synthetic track in a 
television picture, wherein the synthetic track represents the 
path of a movable, natural object in the television picture 
during a given period 6, and wherein a method is employed 
for manipulating the movable, natural object in the televi- 
sion picture. 

In television broadcasts where a movable, natural object 
plays a central part in the broadcast, it can often be difficult 
to follow the object or the movement thereof in the televi- 
sion picture. This is the case, e.g., in sports broadcasts from 
various ball games, such as football, handball, tennis, golf 
and ice hockey, where the picture format used, the back- 
ground of the picture, colour, light conditions etc. can make 
it difficult to follow the object or the movement thereof. The 
object may also be invisible for a brief or a considerable 
period because it is masked by other objects in the picture. 
Such problems can be a factor in influencing the popularity 
of the television broadcasts, which in turn can have conse- 
quences for the sponsors' choice of programs or purchase of 
commercials in such programs, since the viewing figures are 
not as high as is desirable. 

In order to make a game like ice hockey more attractive 
to the television medium, on the basis of the usual complaint 
about television coverage of ice hockey matches that the 
puck is difficult to follow, it has been proposed to make the 
ice hockey puck more clearly visible in the television picture 
by employing a special puck which is equipped with a 
number of infrared emitters driven by a battery provided in 
the puck. On the rink the puck is tracked by sensors placed 
along the edge of the rink, the sensors transmitting infor- 
mation to a camera which is connected to a computer. The 
information is fed to a data processing centre where a signal 
is generated which is processed graphically and introduced 
into the television picture. The puck can thereby be caused 
to change shape or colour. This system has been introduced 
by the company Fox Sports in the USA and is employed to 
represent the puck, e.g., surrounded by a shining halo in 
some colour or other and equipped with a coloured "comet 
tail" when the puck is in motion, e.g. at a certain speed. The 
colours can, however, be altered according to the wishes of 
the producer. In a practical embodiment of this system which 
is called "FoxTrax", 16 sensors are employed around the 
rink and two infrared cameras which also follow the puck. 
The processed representation of the puck is introduced into 
the television pictures which are recorded by the normal 
television cameras, and the special colour effects such as the 
halo and the "comet tail" are overlaid the television signal 
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with a delay which is not greater than between Vfe and 'A of 
a second. "FoxTrax" is described in U.S. Pat. No. 5,564,698. 

However, the system is complicated and offers limited 
opportunities for manipulation of the object, i.e. the picture 

5 of the puck. Another drawback is that the infrared emitters 
in the puck are driven by a battery which has a limited life, 
with a life of only 10 minutes having been quoted. This, 
however, is sufficient to allow the puck to be replaced during 
change of sides in a period of an ice hockey match, but 

10 makes a similar system less suitable in games where the 
object, i.e. the puck or ball, is in play for a longer period. 

U.S. Pat. No. 4,675,816 (Brendon & Vinger) discloses a 
method for electronic localization of the ball in American 
football. The object of the method is to determine whether 

]5 the ball has moved 10 yards forwards and provides simul- 
taneous precise localization information about the ball and 
the possibility of positioning the football on the field. This 
method is regarded as an aid for spectators, officials and 
television. It comprises steps for providing a radio transmit- 

20 ter in the football and transferring radio signals from the 
football to a number of rotating receiver antennas, the 
antenna being directed towards the radio transmitter in such 
a manner that the radio waves supply accurate angular 
direction signals which can be used to calculate the position 

25 of the radio transmitter and the football on the field. These 
angular direction signals are supplied to a microprocessor 
which calculates the position of the transmitter and the 
football, with the use of a triangulation method which 
includes a computation stage which also indicates how far 

30 the ball has moved forwards, and the results of this com- 
putation stage can be shown on display units with a view to 
spectators and television viewers as well as being transferred 
to a control unit which is employed by the officials in order 
to monitor the course of the game. 

35 From U.S. Pat. No. 5,138,322 (Nuttel) there is also known 
a system for continuous and precise measurement of the 
positions of a generally symmetrical object, such as a tennis 
ball, which is in motion in a predetermined three- 
dimensional area, such as a tennis court. In this case a 

40 number of antennas are employed which transmit radar 
signals to the three-dimensional area, reflected return signals 
from the ball being detected and compared with the trans- 
mitted signals for phase determination of the return signals, 
thus enabling unambiguous distances to the object or the ball 

45 to be determined. For this purpose a statistical method is 
employed in order to achieve an accurate determination of 
distance. The path of the object or the ball can be calculated 
simultaneously, and the system is calibrated by placing 
signal reflectors in different fixed positions on the court. A 

50 Doppler radar technique is employed. 

Furthermore, in U.S. Pat. No. 5,346,210 (Utke & al.) there 
is disclosed an object localisation system, especially for 
localizing the ball in a special playing situation in American 
football. The system employs three sensors placed on one 

55 side of the field and a calibration source placed on the other 
side. The calibration source transmits an ultrasound signal 
which is received by the sensors and a ball marking unit 
which can be placed on the field instead of the ball also 
transmits an ultrasound signal which is received by the 

60 sensors, together with an RF signal which is received by the 
calibration source in order to switch it off. The sensors emit 
signals which are used by a processing unit to calculate time 
delays in order to determine the ball's position. 
Alternatively, the ball marking unit may only transmit an RF 

65 signal which is received by the sensors which in turn emit 
signals which are processed in order to determine time 
delays between the receipt of the signals in the sensors. In 
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addition, an automatic ball marking unit which is moved on 
a track is controlled by the processing unit in order to create 
automatically a visual representation of the position of the 
ball. 

None of the above-mentioned, known systems, however, 5 
is particularly well suited to achieve full freedom to manipu- 
late the picture of a natural object in a television picture, all 
requiring the use of relatively expensive and complicated 
systems. Nor are they suitable for all types of games or 
sports and moreover they appear to be substantially i° 
restricted to use in sporting events, but other forms of 
television broadcasts are, however, conceivable, where it 
will be equally interesting to be better able to visualise a 
movable object in a television picture with suitable detection 
and processing methods. 15 

SUMMARY OF THE INVENTION 

Consequently, it is a first object of the present invention 
to provide a system for manipulating the picture of at least 
one movable, natural object in a natural television picture in 20 
such a manner that the object's position and movement are 
clearly visible in the television picture. 

A second object of the present invention is to provide a 
synthetic representation of the natural object, thus enabling 
the synthetic representation to appear as a synthetic object in 
the television picture and to represent the natural object's 
movement and position. 

A third object of the present invention is to manipulate the 
synthetic object in the television picture with regard to shape 3Q 
and colour in such a manner that the viewer will have no 
trouble in following the object. 

A fourth object of the present invention is to determine the 
path of a movable, natural object and to visualise this path 
in the form of a synthetic track for the object in the television 35 
picture. 

Finally, it is an object of the present invention to provide 
a system which makes it possible to detect the movement of 
a natural object and to process the detected data ii 



order to generate and manipulate a representation of the 40 the object, together with 
movable object in a television picture as well as generating 
a synthetic track which represents the movement of the 
natural object in the television picture. 

The above-mentioned objects and other a 
achieved with a method which according to the ii 
characterized by detecting the distance between the object 
and at least 2 fixed basic positions in a preselected x,y,z 
co-ordinate system at a time t, each basic position corre- 
sponding to a known position of a detector, determining an 
x,y,z co-ordinate for the object in the preselected x,y,z 5 
co-ordinate system at the time t, determining the distance 
between the camera's lens centre and the object at the time 
t as an object vector in the preselected co-ordinate system, 
determining the television camera's optical axis in the 
preselected co-ordinate system at the time t as a camera 5 
vector in the preselected co-ordinate system, determining a 
line from the television camera's lens centre to the point of 
intersection between the edge of the generated television 
picture and the plane formed between the object vector and 
the camera vector at the time t as a zoom vector, the object 6 
vector being located between the camera vector and the 
zoom vector when the object is visible in the television 
picture at the time t, and when the object is not visible in the 
television picture at the time t the zoom vector is located 
between the camera vector and the object vector, and deter- 6 
mining an X,Y position of the object referred to the televi- 
sion camera's picture plane and the camera vector on the 



4 

basis of the object vector and the camera vector at the time 
t, and if the object vector is located between the camera 
vector and the zoom vector, to insert a synthetic object in the 
X,Y position in the television picture at the time t, the 
synthetic object constituting a representation of the natural 
object recorded by the camera at the time t, or if the zoom 
vector is located between the camera vector and the object 
vector at the time t, to insert a symbol in the television 
picture, the symbol indicating the location of the X,Y 
position of the natural object outside the edge of the picture 
referred to the television camera's picture plane and the 

The above-mentioned objects and advantages are further 
achieved with a method which according to the invention is 
characterized by calculating the path of the natural object on 
the basis of detected positions x,y,z for the natural object in 
a preselected x,y,z co-ordinate system at the time t, where 
te8, converting the detected positions at the time t to an X,Y 
position in the television camera's picture plane at the time 
t, and generating the synthetic track in the television picture 
as the connecting line between all X,Y positions for the 
natural object in the picture planes of the natural television 
pictures which are recorded sequentially during the period 6. 

The above-mentioned objects and advantages are further 
achieved by implementing the stated methods according to 
the invention with a system which according to the invention 
is characterized in that it comprises a transponder provided 
in the natural object and arranged to react to an optical, 
acoustic or electromagnetic signal received by the transpon- 
der with transmission of a response signal, at least one 
position module with at least 2 position detectors for trans- 
mitting optical, acoustic or electromagnetic signals and 
receiving response signals from the transponder and pro- 
vided in respective basic positions in a preselected x,y,z 
co-ordinate system, together with a signal processor 1 
arranged to determine the distance between a position detec- 
tor and the object at a time t, a computing module connected 
with the signal processor and arranged to compute the x,y,z 
co-ordinates for the object in the preselected co-ordinate 
system at the time t and on the basis of the computed x,y,z 
number of times t to calculate a 



object vector given by the 
distance between the camera's lens centre and the object at 
the time t, a camera vector given by the camera's optical axis 
at the time t and a zoom vector between the camera's lens 
centre and the point of intersection between the edge of the 
picture and the plane formed between the object vector and 
the camera vector at the time t, the object vector being either 
located between the camera vector and the zoom vector or 
the zoom vector between the camera vector and the object 
vector, a camera control system connected with the com- 
puting module and arranged to detect or generate values for 
the camera settings, and a manipulator module connected 
with the camera control system and the computing module 
and arranged to a) create a synthetic object in an X,Y 
position for the natural object in the recorded television 
picture at the time t, the synthetic object constituting a 
representation of the natural object recorded by the camera 
at the time t or b) create a symbol in the television picture, 
the symbol indicating the X,Y position for the natural object 
outside the edge of the picture at the time t, c) generate and 
select attributes for the synthetic object, or d) generate at the 
time t a synthetic track in a recorded or generated television 
picture, the synthetic track representing the path for the 
natural object during a period 6 before or up to the time t. 
BRIEF DESCRIPTION OF THE DRAWINGS 

ill now be discussed in more detail in 
nbodiments and with reference to the 
accompanying drawing, in which: 
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FIG. 1 illustrates a design of the system according to the in a camera K is represented by a vector called the camera 

present invention, e.g. in connection with a ground for a ball vector V^, e.g. for camera IC,, V^. The camera vector 

game, passes through the camera's lens centre and the picture 

FIG. 2 is a block diagram of components in a device for fie u ld ' s geometrical centre axis. The cameras K K ? , K 3 

processing the detected signals in the system according to 5 which are used naturally have zoom lenses and the picture 

f! 6 . . fields can thereby vary in size according to how the cameras 

the present invention, K ^ ^ ^ arg zoomed Mong me edge of me rink tran _ 

FIGS. 3a, b, c, d are a representation of the object in the sponders T K are provided in fixed positions for calibration of 

picture plane of a television camera, the position detectors D 1; . . . D„. Corresponding transpon- 

FIGS. 4a, b, c, d are a representation of the object in the w ders may furthermore also be provided both in the position 

picture plane of a second television camera, where the object detectors D lt . . .D A and the cameras K 1; K 2 , K 3 . 

is partially located in positions outside the picture plane, The connecting line between the lens centre in a camera 

FIG. 5 is a block diagram of a transponder employed in K and the object N which may be a ball or puck or any other 

the system according to the present invention and based on movable object, is called the object vector and designated by 

microwave technique, 15 V^, V WA . 2 , e.g., being the object vector for the camera K 2 at 

FIG. 6 is an active transponder for use with the invention the time 1 and V W 2 the ob j ect veclor for lhe camera K 2 at 

and based on an acoustic surface wave component, the time The same applies to the other cameras. The 

FIG. la is a polling signal emitted in the form of pulses distance [ ] om a P° sition df f cl ° r D t0 the ob Jf N at ll f 

from a position detector, ' im6 u l ls designated as a and at the time t as a , for example 

. • , . r> £ , 20 for tne position detector D, as a, and a', respectively. 

FIG. 76 is a response signal emitted in the form of a pulse zu r ,,. ,. 1 . 1 , . .. ... 

j. .. ." „ . When a polling signal is emitted from a position detector 

sequence from the acoustic surface wave component, ^ . F . 5 ? , • , ■ - Jt 

A, r ^ . ■ ■ D, after a certam period x a response signal is received from 

FIG. 8 is a system according to the present invention (he natural object N ^ nse si al be a rcflection 

implemented for detection of several objects which are m rf ^ poUing pulse Qf it fflay be a response pu[se frQm [he 

motion m separate or connected areas, 2; oh - £Ct N> the response pulse being triggered by the polling 

FIGS. 9a, b, c illustrate generation of a synthetic track by signa i from the position detector D. The time between the 

means of the design according to FIG. 1, transmission of the polling signal and receipt of the reply 

FIGS. 10a, b illustrate generation of a synthetic track by signal, i.e. x, now becomes a measurement of the distance a 

means of the design according to FIG. 8, and between a position detector D and the object N. For 

FIGS. 11a, b illustrate generation of synthetic tracks for 30 example, the position detector Dj finds that the distance to 

two objects by means of the design according to FIG. 8. N at the time t is a 1; and at the time t 1 a.\. The same applies 

to the remaining position detectors. If the object N is now 
located all the time on the x,y plane in the area A, its position 
will be unambiguously determined by using two position 

FIG. 1 illustrates the present invention realised at a sports 35 detectors, e.g. T) 1 and D 2 and finding respective distances a^ 

centre, such as a ground for ball games or an ice hockey rink a'^ a 2 , a' 2 . If the area Ais three-dimensional, i.e. the position 

A, where the area A is the projection in an x,y plane of the to the object N is determined by the co-ordinates x,y,z at the 

area which has to be covered by position detectors D 1; . . . time t, it is necessary to have at least three position detectors, 

D„. The detectors are connected via a signal line L to a data for example D„ D 2 , D 3 . As illustrated in FIG. 1 four position 

processing unit Q. The data processing unit Q is further 40 detectors D 1; . . . D 4 are preferably used to achieve unam- 

connected via a local data bus B L to a regulator module R 1; biguous detection of the position of the natural object N. The 

R 2 , R 3 which is assigned to respective television cameras position of the object N is thereby solely determined by 

Kl, K 2 , K 3 and comprises not shown bodies, preferably in distance measurements, i.e. by trilateration, a method which 

the form of servos, for setting the cameras together with not is well known to those skilled in the art and therefore will not 

shown sensors for detection of the camera settings, includ- 45 be discussed further here. Thus, as distinct from distance 

ing camera angles and zoom angles. The position detectors determination by triangulation, it is not necessary to deter- 

Dj, . . . D„ comprise as shown antennas H, and these are mine direction angles to the object N. Moreover, in the 

directive, with their main lobes m^ substantially covering actual position determination a statistical optimization may 

the area A and a certain height above the plane of the area advantageously be employed in order to reduce any posi- 

A. In FIG. 1 the main lobe for the position detector D t is 50 tioning errors, such as estimation by means of the least 

indicated by broken lines M D1 . Both the position detectors squares method. 

D 1; . . . D 4 and the television cameras K 1; K 2 , K 3 are As illustrated in FIG. 2, according to the invention the 
positioned in a preselected x,y,z co-ordinate system, since system comprises a transponder T provided in the natural 
the plane of the rink A as mentioned may lie in the object N. The transponder T may be a passive transponder, 
co-ordinate system's x,y plane, the z axis then being per- 55 such as a reflector for microwave signals or radar signals 
pendicular to the x,y plane (orthogonal co-ordinate system). transmitted by the antenna H in the position detector D or it 
The positions of both the position detectors D 1; . . . D 4 and may be an active transponder which is triggered by the 
the cameras K 1; K 2 , K 3 are precisely defined in the x,y,z polling signal transmitted by the position detector D and 
co-ordinate system. In the area A, i.e. on the rink or over it emits a response signal which is detected by the position 
at a certain height there is located a movable object N which 60 detector D. The position detector D with antenna H forms 
is illustrated in FIG. 1 in two positions x,y,z and x',y',z' at the part of a position module M which comprises a signal 
times t and t' respectively. In other words, during a time processor 1, the signal processor 1 preferably being pro- 
interval 9=t'-t the movable object has moved from the vided in a data processing unit Q. The signal processor 1 is 
position x,y,z to the position x',y',z'. The cameras K 1; K^, K 3 connected via the signal line L with, for example, two or 
are directed towards the area A and are set so as to cover a 65 more position detectors D lf . . . , finding the detection 
picture field of a certain size, in FIG. 1 for each camera distance a on the basis of the measured running times x. For 
indicated by broken lines. The picture axis or the optical axis this purpose there will be provided in the signal processor 1 
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components which are well known to those skilled in the art, 
including a clock with a very high clock rate, the clock rale 
at least being adapted to the frequency for the polling signals 
emitted from the position detectors D. The detected dis- 
tances a are given to a computing module 2 which by means 
of trilateration calculates the positions x,y,z at different 
times t and thereby also the path of the object N on the basis 
of positions detected at the different times t. The computed 

positions are supplied to a camera control system 3 in the 

data processing unit Q. In the camera control system 3 w K^for~ exampKrecords 
values are calculated for the settings of the camera K by known camera angle and 
means of a regulator module assigned to the camera K, and 
the camera settings can be controlled automatically by 
means of a control loop provided in the camera control 
system 3, on the basis of the existing camera settings and 
detected positions for the object N. The optical system 
(objectives and other lenses) in the camera K are indicated 
schematically and designated by Q. 

The regulator module R comprises not shown servos for 
generating the camera settings and not shown sensors for 
detection and recording of the camera settings. One of the 
sensors is an angle sensor for determining the camera angle, 
and use is preferably made of a triaxial angle sensor which 
thereby indicates the direction of the camera's optical axis or 
the camera vector V^. The angle sensor can be calibrated by 25 the object 
measuring the angle of direction at a fixed point. Such angle 
sensors are well known to those skilled in the art and 
therefore do not require further mention here. The camera K 
can naturally also be manoeuvred manually in order to set 
zoom angles and camera angles. 

The data processing unit Q also comprises a manipulator 
module 4 for generating a synthetic object S which corre- 
sponds to the natural object N, the fixed position of the 
natural object being converted and scaled to a corresponding 
(projected) position X,Y in the picture plane of the camera 
K, taking into account the recorded values for the camera 
settings. Thus in the position X,Y in the camera's picture 
plane, via, e.g., a video generator (not shown) provided in 
the manipulator module 4, a synthetic objf 



ing unit Q is employed, the production unit P and the data 
processing unit Q may preferably be integrated into one unit, 
but if a plurality of data processing units are provided, the 
system may be completely decentralised, with all the data 
s processing units thus being connected to the production unit 
via the global data bus B G . 

The representation of the synthetic object S in a television 
picture will now be described in more detail with reference 
to FIGS. 3 and 4. As illustrated in FIGS. 3a and 3b, a 
a known positi 
m angle ; 




picture in the picture plane l K2 . The camera 
normally located on the picture plane I^ 
thereof. The natural object N which at the tii 
in the x,y,z position has the object vector 
an angle with the c; 

lens centre co. As can be seen in FIG. 3b, the tv 
and V NK2 form a plane which intersects the edge of the 
picture plane l K2 at the point 2^. The connecting line 
between the lens centre w and the point Z 2 is called the zoom 
vector V Z2 , as illustrated in FIG. 3b, and is thereby deter- 
mined by the camera settings. The representations of the 
remaining vectors in the optical system Q are marked with 
star, for example V* K2 for the camera vector and V*^ for 
~ tor V Z2 also forms an angle 

with the camera vector W* K2 as illustrated in FIG. 36. If the 
angle between the zoom vector V 22 and the camera vector 
V *K2 ' s greater than the angle between the object vector 
y* NK2 and the camera vector V*^, the natural object is 
reproduced in the picture plane I A . 2 and in FIG. 3a illustrated 
represented by a star-like object which constitutes the syn- 
thetic object S. In FIG. 3c the natural object N has moved to 
the position x',y',z' at the time t' and the vector parameters 
for the dynamic vectors, i.e. the object vector V N and the 
zoom vector V z has changed, so that they are illustrated in 
FIG. 3c as V' AX2 and V' Z2 respectively. The same applies to 
FIG. 3d which shows the representation of the vectors in the 
optical system and is marked in the same way as in FIG. 3b. 
The natural object N is still in the picture field and can be 



created, representing the natural object N in its position X,Y 40 represented by the synthetic object S. In FIG. 4 the picture 



in the picture plane in the camera K, and the synthetic object 
S can be represented with various attributes for size, shape 
and colour. The primary object here is that the synthetic 
object S which represents the natural object N should at all 
times display the position and/or the movement of the 4 
natural object N, as it would be represented in the television 
picture at any time t. 

The signal processor 1, the computing module 2, the 
camera control system 3 and the manipulator module are all 
interconnected via the local data bus B 1( which also passes 5 
signals to and from the camera's regulator module R. If a 
plurality of television cameras K are assigned to the data 
processing unit Q, they are also naturally connected to the 
local data bus B L . From the data processing unit Q a global 
data bus leads to a production unit P which takes care of 5 
the actual production of the television broadcast and may 
transfer the pictures to the television station or the television 



0 different times t and t' for the 
t the position x,y,z of the natural 



In the production unit P there are displayed the natural 
television pictures recorded at any time, or possibly syn- 
thetically generated television pictures with a synthetic 
object S and the recorded television pictures with the 
synthetic object overlaid in the correct position, thus 
enabling the producer to select the camera and the picture 
which is desired to be represented at any time during a 
television broadcast and transmit the pictures via a standard 
TV line to the television transmitter. If only a data process- 



1 shown a 
camera K 3 . At the time t 

object falls outside the camera's picture field. As illustrated 
in FIG. 3b the angle between the camera vector V* m and the 
object vector V* NK3 is greater than the angle between the 
zoom vector V B and the camera vector V*^ 3 . In other 
words the object is outside the edge of the picture. This may 
be advantageously indicated in the television picture by 
placing the synthetic object S or an indicator therefor at the 
edge of the picture, i.e. at the point Z 3 , the line between the 
centre of the picture and the synthetic object indicating the 
direction of the synthetic object. Naturally, a synthetic 
direction indicator can also be employed to point to the 
object's position outside the edge of the picture. In FIG. 4b 
which illustrates the picture plane 1'^ at the time t' the 
movable or natural object N has moved into the picture field 
and can be represented by the synthetic object S as shown, 
the angle between the object vector V'*^ and the camera 
vector V'*^ now being smaller than the angle between the 
zoom vector V Z3 and the camera vector V 1 *^, as can be seen 
in FIG. Ad. 

As mentioned, the attributes of the synthetic object S may 
be freely chosen and the object naturally does not need to be 
a true representation of the natural object, either with regard 
to size, shape or colour. An ice hockey puck, e.g., may be 
generated as a highly luminous, pulsing or blinking object in 
a contrasting colour. Moreover, the size or colour of the 
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object can be caused to vary in order to indicate to the viewer 
the apparent distance between him and the natural object, as 
it appears when viewing a television picture. Furthermore, 
the synthetic object can be provided with labels or 
indicators, e.g. of an alphanumerical nature or in the form of 
shining or blinking arrows or other visual indicators in order 
to indicate the distance of the object from, e.g., the camera, 
or also the speed, direction and position of the object. All of 
this can be introduced into the shown television picture in 
connection with the synthetic object S and overlaid the 
natural background in the television picture. The actual 
generation of the synthetic object and other indicators is 
performed in the manipulator module shown in FIG. 2, and 
may, e.g., take place by means of a not shown video 
generator of a type which is well known to those skilled in 
the art. 

The detection of the natural object and the distance 
determination will now be described in more detail. 

There are known and described in the art a number of 
different systems for measuring the distance to an object, 
e.g. by transmitting from an antenna optical, acoustic or 
electromagnetic signals which are reflected back to the 
antenna by the object whose distance has to be determined. 
The measured time difference between the transmission and 
receipt of the signals is a direct measure of the distance to 
the object, since the propagation velocity of the signals in 
the surrounding medium is assumed to be known with 
sufficient accuracy, and a time measuring system is 
employed which provides the desired accuracy in determi- 
nation of the distance. For example, in U.S. Pat. No. 
3,503,680 (Schenkerman) there is disclosed a distance mea- 
surement system based on a pulse radar. The technique 
therein disclosed takes into account the fact that the object 
whose distance has to be measured can move at a relatively 
great speed, with the result that instead of transmitting a 
pulse and then measuring the running time of the pulse 
between the object and the antenna, a sequence of pulses is 
used, the receipt of the echo or the return pulse of a 
transmitted pulse being employed to trigger the transmission 
of a second pulse. This process is repeated until a predeter- 
mined number of echo pulses has been received, and the 
time taken to receive a predetermined number of echo pulses 
is proportional to the distance to the object. The method can 
be employed both with electromagnetic signals and acoustic 
signals. 

GP-PS no. 1,290,915 (Allard & Clark) also discloses a 
distance measurement system based on pulse radar of a 
similar nature to that disclosed in the above-mentioned U.S. 
patent. 

Otherwise it is well known in the art to measure the 
distance to a movable object by means of a so-called CW 
radar which finds the radial speed of the movable object by 
means of a detected Doppler shift in the return signal or the 
echo signal. By employing a phase comparison it will be 
possible to find the distance to the movable object. 

The disadvantage of using distance measurement systems 
based on detection of a return echo, however, is the many 
sources of error which can arise, e.g. due to false and 
spurious echoes and so-called glitter, i.e. reflected noise 
from ground or sea, and similarly the fact that the natural 
object concerned may not be suited to return an echo, even 
though it could conceivably be equipped with radar reflec- 
tors or a reflecting surface. In various kinds of sport, such as 
ice hockey, however, this is assumed to be an unsuitable 
method. 

Instead of using a passive transponder, i.e. a return echo 
or reflected signals from the natural object, according to the 
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present invention use is preferably made of an active tran- 
sponder. In this case an active transponder should be under- 
stood to refer to a transceiver which is provided in the 
natural object and which, on detecting a polling signal, itself 

5 transmits a response signal. As a rule the transceiver must 
have its own energy source and a pulse transmitter for 
generating the response signal, and in this connection an 
energy source, e.g. in the form of a battery will be a 
substantial drawback, both because the battery can be dam- 

1Q aged and because it will have a limited life. Since the active 
transponder also has to be mounted in the natural object, it 
has to be robust and capable of withstanding jolts and shocks 
as well as relative high accelerations. In this context refer- 
ence may be made to the stresses to which, for example, a 
tennis ball, a golf ball or an ice hockey puck will be exposed 

15 when they are struck by, e.g., a racket or club. 

There is therefore a requirement for the active transponder 
to be robust, to withstand considerable acceleration stresses 
and not to require its own energy source, while at the same 
time being small enough to be inserted in a natural object 
which at any rate is not larger than an ice hockey puck. A 
transponder of this kind should also be suitable to be worn 
by, e.g., individuals without causing them any 
inconvenience, such as participants in a game or athletes or 
other people whom it is desirable to record in a television 
picture by means of a synthetic representation of the person 
concerned. This will be discussed in more detail later. 

A block diagram of a transponder as used according to the 
invention in the natural object is illustrated in FIG. 5. The 

30 transponder consists of an antenna 5, an impedance match- 
ing network 6 and a pulse transmitter 7. 

As stated, the transponder's pulse transmitter can be 
implemented as an active unit based on battery operation or 
as a passive system without a battery. In the present 

35 invention, however, an active system without battery is 
preferably employed which has, as already mentioned, a 
number of operative advantages and this is preferably imple- 
mented by means of acoustic surface wave technique (SAW 
technique). 

40 The transponder as illustrated in FIG. 5 and used in the 
present invention is therefore a so-called acoustic surface 
wave component. Such acoustic surface wave transponders 
or SAW transponders consist of a crystal plate with an input 
electrode and one or more output electrodes, as is illustrated 

45 and will be described in more detail in connection with FIG. 
6. 

An SAW transponder T as illustrated schematically in 
FIG. 6, has long been known to those skilled in the art in and 
consists in principle of a crystal, e.g. of lithium niobate with 

50 a surface pattern of metal which constitutes transducers, 
reflectors, etc. A polling pulse from the position detector D 
is received by a transducer 8 which is shown in the form of 
a so-called interdigital transducer. The received electromag- 
netic energy in the polling pulse is converted in the trans- 

55 ducer 8 to an acoustic surface wave which moves along the 
crystal. At a certain distance from the transducer 8 there are 
placed a first reflector 9a and a second reflector 9b respec- 
tively. When the acoustic wave strikes the two reflectors 9a 
and 9b, reflection waves are created which move back to the 

60 transducer 8. The transducer 8 will convert the two acoustic 
reflection waves to electromagnetic pulses which constitute 
the response signal which is transmitter via the transpon- 
der's antenna. At the end of the transponder T there may be 
provided a surface wave absorber 10. The path of the signal 

65 is illustrated schematically in FIGS, la and 7b. 

When the transponder T receives an interrogator or poll- 
ing pulse 11 of length % p , as illustrated in FIG. la, the pulse 
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transmitter 7, i.e. the transducer 8, is triggered, and a 
response signal is transmitted by the antenna after a specific 
time x D . As illustrated in FIG. lb, the response signal 
consists of two pulses 13a, 136, located at a distance T x apart 
from each other. For different categories of natural objects 
the category of the object may be determined by, e.g., 
measuring the pulse distance in the position detector D 
which also transmits the polling signal. 

In principle this technique is exactly the same as tech- 
niques for the use of radar transponders based on SAW 
technique, since, as is well known, SAW components are 
used as delay members in RF communication and detection 
systems. The transponder, which may be in the form of an 
integrated enclosed chip, can be coded by selecting a pulse 
distance between two reflection pulses equal to an integral 
multiple of, e.g., the length x p of the polling pulse. The 
transponder T may, of course, comprise more than the two 
reflectors 8a, 86 as illustrated in FIG. 6, and in this case it 
can be used to emit the response signal with a large number 
of different codes. In this way a transponder T based on 
SAW technique can be employed in detection of the distance 
to a large number of natural objects N, since the code thus 
unambiguously identifies which of the natural objects are 
involved. This may be relevant in cases where the method 
and the system according to the present invention are 
intended to monitor a game where several objects are 
employed simultaneously in the game, such as in golf or 
during athletics competitions where it is desirable to follow 
a large number of competitors each of whom is equipped 
with a transponder. In the case of approximately simulta- 
neous detection of several response signals, in order to 
achieve an unambiguous detection it may be appropriate to 
use special detection techniques, e.g. based on correlation 
between detected, coded reply pulse sequences and pre- 
stored code sequences for each individual transponder. A 
second possibility is to use polling signals at different 
frequencies and corresponding frequency-tuned transpon- 
ders. Finally, transponders may be employed with different 
delays x rl . In this case the delays must be adapted to the 
distance range concerned for detection, so that the response 
signals for the detector in respective unambiguously defined 
times are relative to the time for the transmission of the 
polling signal. 

FIG. la illustrates a pulse pattern employed in the posi- 
tion detector D, where polling pulses 11 are transmitted with 
pulse length x p and with a given time interval between the 
polling pulses corresponding to a desired sampling fre- 
quency sequence. The response signal from the transponder 
T will arrive at the position detector D in the form of the 
pulse sequence 13a, 136, as illustrated in FIG. lb, while in 
this case the pulse shape 11 represents an echo caused by 
ground or sea glitter and centred around the real distance to 
a reflection point at or near the transponder T. However, use 
of a SAW component as transponder will delay the response 
signal by a predetermined value x d and the reply pulses 13a, 
136 will therefore arrive some time after the glitter pulse 11 
and thereby not be masked by it. The distance selected 
between the reply pulses 13a, 136 here is x x and the delay 
will be equal to x d , the distance from the transducer 7 to the 
reflector in this case naturally corresponding to a running 
time of Vz x d and in addition the distance between each 
reflector corresponding to a running time of Vi x x . The reply 
pulse from the transponder T will now arrive at the detector 
D after an interval x=x a +x d , wherein x a is the running time 
for an electromagnetic signal from the position detector D to 
the transponder T and back and x d the time delay entered in 
the transponder. To the period x d there will naturally be 
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added further periods x x for the reply pulses from each 
further reflector 9 in the transponder T. What is now required 
of the system is that the time x should be able to be measured 
with an accuracy which is preferably of the order of 10" 1 m, 

5 possibly even less, and this will be within the scope of 
current technology. Thus the time delay x d and the period x x 
must be precisely known, and this can be achieved by a 
careful calibration of the SAW transponders used. In order 
to achieve such accuracy, the frequency selected for the 

10 polling signal should be at the upper end of the L band or in 
the S or X band, i.e. it should have a frequency of between 
circa 1 Mhz and 10 Mhz, or in other words a wavelength of 
between 30 cm and 3 cm. The clock employed in the signal 
processor 1 should have a rate which is adapted to the 

15 frequency of the polling signal and the response signal. Thus 
with present day technology distance detections can be 
obtained with an accuracy of the order of 10" 1 m or less 
within the distances which will normally occur when using 
the method and the system according to the invention, i.e. 

20 preferably in sports arenas or the like. 

In order to test and calibrate the system according to the 
present invention, in the corner of the area A as illustrated in 
FIG. 1 there can be provided permanently mounted and 
accurately calibrated transponders T K in fixed positions and 

25 coded in order to emit unambiguous identifying response 
signals. These are used to calibrate the system and the 
position detectors D. Transponders are also preferably pro- 
vided both in the position detectors and the cameras. When 
mobile cameras such as "Handycam" and "Steadycam" are 

30 used, it will be possible to determine the camera positions by 
means of the position detectors D. 

It will also be seen that by using the SAW transponders as 
preferred in the present invention, it will be possible to place 
these without difficulty in most natural objects N which it 

35 will be appropriate to follow and display in connection with 
a television broadcast, whether it be a football or an ice 
hockey puck. As mentioned, SAW transponders are very 
robust and capable of withstanding substantial acceleration 
stresses, nor do they require their own power source, while 

40 at the same time they can be designed with extremely small 
dimensions, measuring, for example, only a few millimeters. 
They may, e.g., be enclosed in a shock-absorbing material 
fitted inside the natural object, but otherwise no special 
measures are necessary to protect them from damage. At the 

45 same time it will be extremely simple to replace them if they 
really should be destroyed. However, they are also so 
inexpensive to manufacture that it will be just as easy to 
replace the natural object with another with a similar SAW 
transponder. When used in connection with other natural 

50 objects than, e.g., footballs, ice hockey pucks etc., for 
example worn on a person, the transponders may be 
designed in the form of an enclosed chip which is attached 
to the person and worn during the period concerned. Since 
the SAW transponder can be arranged to emit an unambigu- 

55 ous coded reply signal, the person wearing the transponder 
chip will also be unambiguously identifiable. 

The system according to the invention may comprise a 
number of position modules M which are assigned to one or 
more respective television cameras K, each position module 

60 being arranged to cover a predefined area in the x,y,z 
co-ordinate system. This implies that the use of the method 
and the system according to the present invention is not 
restricted to only a well-defined area such as a field for ball 
games or an ice hockey rink and the like, but can be 

65 employed to cover areas of arbitrary shape and size, such as, 
e.g., cross-country skiing courses, Alpine skiing slopes and 
the like. Nor are the method and the system according to the 
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present invention only restricted to sporting arrangements, 
but they can be used to cover events within large and 
relatively freely defined areas. Such areas can be divided 
into several partial areas, and the system does not necessar- 
ily have to cover each partial area or the entire area, but the 5 
use of the system may be restricted to selected partial areas. 

An example of a set-up of the system according to the 
present invention for use, e.g., in a ski run or Alpine skiing 
slope is illustrated in FIG. 8. Here, the area A is divided into 
three partial areas A x , A 2 , A 3 . To each partial area A 1; A 2 , A 3 10 
there are assigned respective position modules Mj, M 2 , M 3 . 
The partial area A 2 is covered, e.g., by a position module M 1 
with 3 position detectors Dj, D 2 , D 3 which are intercon- 
nected via the signal line L 2 and thereby connected to a data 
processing device Q r The data processing device Qj is 15 
connected via a local data bus B 1 with the cameras K J; K 2 
via their respective regulator modules R 1; R 2 . Typical pic- 
ture fields for the cameras K 1; K, are indicated by broken 
lines and similarly are surrounded by the main lobe of the 
antenna in one of the position detectors D 3 indicated by 2Q 
broken lines m 03 . The data processing device Qj is con- 
nected to a production unit P via a global data bus B G . The 
same applies to position modules and cameras which are 
assigned to the remaining partial areas A 2 ,A 3 . Moreover, in 
each of the partial areas A 2 , A 2 , A 3 transponders T^ are 25 
provided for calibrating in fixed positions, such as at the 
edge of each partial area. Furthermore, the global data bus 
B G may be connected to the timekeeping system, indicated 
in FIG. 8 as CL, at "START" and CL 2 at "FINISH" 
respectively. 30 

In FIG. 8 two natural objects N lr N 2 are illustrated, the 
first natural object N x being substantially located in the third 
partial area A 3 , and the second natural object N 2 in the 
partial area A 1 . Each of the objects is shown in two different 
positions x,y,z; x',y',z' at the times t and t' respectively, where 35 
it should be understood that positions and times do not need 
to be identical for the two objects N 1; N 2 . The path of each 
of the objects N x , N 2 between the first and the second 
position is indicated by a broken line. The distances from the 
position detectors D to the objects N ± , N 2 and their positions 40 
are determined as described earlier in connection with FIG. 
1. 

Since the position modules' polling signal has a high 
frequency and directive antennas are employed, it will be 
understood that the signals move substantially in a straight 45 
line. In other words they can be slopped by obstacles in the 
terrain and the like. By dividing up the area A into sub-areas 
Aj, A 2 , A 3 which are assigned to respective subsystems of 
position modules D and cameras K, the entire area A will 
still be able to be covered by using the method and the 50 
system according to the present invention. The selected 
positions of the position modules D and the cameras K must 
therefore take topographical and other conditions into con- 
sideration. 

In the area A obstacles may be encountered which prevent 55 
the objects N from being shown in the cameras' picture field. 
This may, e.g., be vegetation, groups of people and the like, 
but it will not obstruct the free passage of the detector and 
transponder signals employed. This means that the position 
of the natural object N can still be indicated in the cameras' 60 
picture field, even though the object N at the time concerned 
happens to be covered by obstacles which hinder visual 
contact. In this case the natural object N may naturally be 
represented by a synthetic object S in the correct position 
X,Y in a television picture. Such conditions can easily arise, 65 
e.g., in reports from cross-country skiing, where the skiers 
will be invisible due to the vegetation, even though the 
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camera is located in a position which should in theory cover 
the section of the cross-country course concerned. Thus by 
means of the design illustrated in FIG. 8, the methods and 
the system according to the present invention permits a 
complete coverage to be obtained with indication of the 
position of the natural objects N, in the case of sporting 
events or naturally also the position of the competitors, even 
though they are, e.g., masked by vegetation and the like in 
the cameras's picture field. Neither reporters nor viewers 
need, therefore, to be left to guess where an anticipated 
favourite is located at that moment. 

As mentioned above, by means of a method according to 
the invention a synthetic track can be generated in a televi- 
sion picture, the synthetic track thus being intended to 
represent the path of a natural object N in the television 
picture during a given period 8. The path of the natural 
object N can be calculated, as already mentioned, on the 
basis of detected positions x,y,z in a preselected co-ordinate 
system at the time t. The time t will thus lie within a period 
6. The detected positions x,y,z are converted to an X,Y 
position in the picture plane of a relevant television camera 
at the time t. The synthetic track in the television picture is 
created as the connecting line between all X,Y positions for 
the natural object in the picture planes of the television 
pictures which are recorded sequentially during the period 0. 
The synthetic track may now, e.g., be displayed in a still 
picture which need not be recorded during the period 0, or 
it can be continuously updated and generated for each 
individual picture which is recorded during the period 6, 
thus creating the synthetic track cinematographically. Like 
the synthetic object S the synthetic track can be created with 
given, possibly similar attributes with regard to size, shape 
and colour. It may preferably be shown, e.g., as a coloured 
line in a contrasting colour and with indicators for direction 
of movement and possibly also the speed of the natural 
object in the path which corresponds to the synthetic track. 

An example of generation of a synthetic track for a natural 
object N is illustrated in FIGS. 9a, 9b and 9c. FIG. 9a shows 
a path a where the natural object moves from positions p 0 to 
p 3 during the period 0. FIG. 9a illustrates this on the 
projection of the path A in an x,y plane. The path of the 
object N during the period 0 is covered by the camera K and 
naturally recorded as a sequence of individual pictures by 
the cameras K. If camera K has the same setting during the 
period 0, the picture field, e.g., of the camera K in the picture 
plane \ K will show a section A 1 of the area A, as illustrated 
in FIG. 9b. Here the natural object N is represented by a 
synthetic object S shown in the relevant positions p 0 , . . . and 
with the projection of the natural object's path projected in 
the picture plane \ K as a synthetic track between the various 
positions p 0 , . . . for the synthetic object S as shown in the 
figure. If, e.g., it is a television report from a football match 
which is being shown, N will, naturally, usually be the 
football and the synthetic track will represent the ball's path 
from the position p 0 to the position p 3 , in three dimensions, 
but projected into the picture plane \ K . I^t) may be the final 
picture in the sequence which was recorded during the 
period T, and in this case the track shown for the synthetic 
object S is the path from the beginning of the period 8 up to 
the last recorded picture during 0. It is, of course, not 
essential that the track should only be shown continuously in 
the recorded pictures during the period 0, since the synthetic 
track can be inserted in a freely selected still picture or in a 
television picture which is recorded at a time outside the 
period 0. 

Nor is it a condition that the synthetic track should only 
be shown in natural television pictures, and, as illustrated in 
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FIG. 9c, the actual television picture may be a synthetic 
television picture I^(S) produced, e.g., by a video generator, 
not necessarily with a view to representing the camera 
perspective, but it may be some kind of graphic represen- 
tation of the area A. FIG. 9c shows a graphic representation 5 
A 5 of the field A reproduced in an X,Y plane in the form of 
a synthetic television picture. The movement of the natural 
object N is then shown projected in the X,Y plane as the 
movement of the synthetic object S from the position p 0 to 
position p 3 . A person skilled in the art will easily see that it 10 
will be possible to use different perspectives, such as a side 
view, or also possibilities of manipulating the picture with 
the synthetic object and the synthetic track by means of 
various video graphic methods. Nor is there any reason why, 
on the basis of the data acquired for the position and the 1S 
movement of the natural object N during the course of the 
game, the path of the natural object should not be presented 
during the course of a game or during a selected period, thus 
aiding the analysis of the game. The technique may also be 
employed by meeting officials and referees in order to judge 20 
situations which otherwise would be difficult to assess if the 
judgement alone were to be based on momentary impres- 
sions during the course of the game. 

As mentioned, there is no reason why the system should 
not detect and monitor a number of natural objects N lt . . . 2 5 
N„, the natural objects being unambiguously defined by the 
use of transponders which emit coded reply signals which 
unambiguously identify the natural object concerned. FIG. 
lis refers to a situation which could arise in an area as 
illustrated in FIG. 8. Two natural objects N have moved 30 
through the partial area A 3 , but during different periods. The 
first natural object N 1; e.g. has moved from the position p 0 
to the position p 3 during the period 6, while the natural 
object has moved from the position p' 0 to the position p' 3 
during the period 0'. The paths of the natural objects are 35 
indicated as broken lines in FIG. 9a. In FIG. 9b the detected 
and calculated paths of the natural objects N-p N 2 are 
represented by the corresponding synthetic objects S 1; S 2 
and converted to synthetic tracks for the objects S 1; S 2 , being 
illustrated simultaneously in one and the same picture I^t) 40 
at the time t. Data for times and speeds which form the basis 
for a comparison between the movement of the objects N 1; 
N 2 , for example Alpine skiers, can be introduced into the 
picture. The X,Y projection of the partial area A3 in the 
picture plane \ K at the time T will appear as the picture I t (t) 45 
illustrated in FIG. 116 and choice of line and skiing style 
can, e.g. be directly compared. 

It will be obvious to a person skilled in the art that it is 
possible to create a real time reproduction, i.e. the 
development, e.g., of a game or race can be followed in 50 
simulated real time by means of synthetic objects and 
synthetic tracks in the television picture. This may occur, 
e.g. in connection with a playback, but when using the 
method and the system according to the present invention, 
the producer of the television broadcast has a great deal of 55 
freedom to manipulate objects and information as they are 
displayed either in natural or synthetic television pictures. It 
will be possible, e.g., to combine synthetic tracks and 
synthetic objects with various forms of animation graphics, 
if so desired. 60 

As shown in FIG. 1 and FIG. 8, transponders T K are 
installed in various fixed positions in the area A for calibra- 
tion of the system. As mentioned above, transponders may 
also be installed both in the position detectors D and in the 
cameras K. In the latter case movable or mobile cameras can 65 
be employed, i.e. cameras which are not mounted in a fixed 
position, and may be of the type "Handycam" or "Steady- 
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cam". These cameras must also naturally be connected to 
data processing devices Q via data buses B, or local data 
buses, in which case they may expediently be based on a 
wireless connection. 

The method and the system according to the present 
invention are not necessarily limited to sports reports, but 
may be applied in a number of other kinds of television 
transmissions. It will also be possible to employ the method 
and the system in transmissions which are not restricted to 
fixed and specific areas, e.g. in connection with nature 
programs on television. If the natural object is located 
beyond the range of the position detectors, but within the 
field of view of a camera, the position of the natural object 
must be determined by other means. In this case the natural 
object may be equipped with a GPS (Global Positioning 
System) receiver for determination of the position, this 
being remotely read at the production location. Similarly, the 
position of a mobile camera, which is also located beyond 
the range of the position detectors, is determined by means 
of GPS and transferred to the production location. Positions 
and paths for the natural object can then be calculated 
according to the method in the present invention, and the 
natural object is represented by a synthetic object which is 
inserted in the correct position in a television picture 
recorded by a camera where the natural object is located in 
the picture field. An example of such an application in nature 
reports may be, e.g., animals which are equipped with radio 
transmitters and GPS receivers and are followed, e.g., by a 
helicopter-borne "Steadycam". Even though the natural 
object may be hidden by obstacles which prevent it from 
being viewed directly in the television picture, its position 
and movement can still be indicated by a synthetic object or 
a synthetic track which is inserted in the television picture. 
In such cases the positioning accuracy is not particularly 
critical and may well amount to several meters, or possibly 
several tens of meters. Such a positioning accuracy lies 
within the range of possibility when using GPS at its highest 
time resolution. 

As stated, in principle the natural objects, which have to 
be detected and displayed represented as synthetic objects 
and possibly represented in motion as synthetic tracks, may 
be practically anything at all. In sports competitions, e.g., the 
competitors may also be equipped with transponders and 
their positions and movement are detected and displayed. 
Since the transponder's reply signal may be in the form of 
an unambiguous code, the identification of the transponder 
will also be unambiguous and the necessary identification 
information can naturally be shown in the television picture 
in some way or other, such as by showing the portrait of the 
competitor overlaid a part of the television picture. 

The use of the method and the system according to the 
present invention in television broadcasts and television 
reports will be capable of being implemented in other 
circumstances than those herein described, and it should also 
be understood that the methods and the system according to 
the present invention for displaying and representing the 
natural object and its movement in the form of a synthetic 
object in a television picture and a synthetic track for the 
synthetic object can be realised in a number of different 
variants and varying attributes which are not expressly 
indicated here, but which nevertheless will be obvious to 
those skilled in the art and which fall within the scope of the 
present invention. 

What is claimed is: 

1. A method for manipulation of at least one movable, 
natural object in a natural television picture, wherein the 
television picture is generated by one or more television 
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cameras, characterized by detecting the distance between the 
object and at least 2 fixed basic positions in a preselected 
x,y,z co-ordinate system at a time t, each basic position 
corresponding to a known position of a detector, determin- 
ing an x,y,z co-ordinate for the object in the preselected x,y,z 
co-ordinate system at the time t, determining the distance 
between the camera's lens centre and the object at the time 
t as an object vector in the preselected co-ordinate system, 
determining the television camera's optical axis in the 
preselected co-ordinate system at the time t as a camera 
vector in the preselected co-ordinate system, determining a 
line from the television camera's lens centre to the point of 
intersection between the edge of the generated television 
picture and the plane formed between the object vector and 
the camera vector at the time t as a zoom vector, the object 
vector being located between the camera vector and the 
zoom vector when the object is visible in the television 
picture at the lime t, and when the object is not visible in the 
television picture at the time I, the zoom vector is located 
between the camera vector and the object vector, and deter- 
mining an X,Y position of the object referred to the televi- 
sion camera's picture plane and the camera vector on the 
basis of the object vector and the camera vector at the time 
t, and if the object vector is located between the camera 
vector and the zoom vector, to insert a synthetic object in the 
X,Y position in the television picture at the time t, the 
synthetic object constituting a representation of the natural 
object recorded by the camera at the time t, or, if the zoom 
vector is located between the camera vector and the object 
vector at the time t, to insert a symbol in the television 
picture, the symbol indicating the location of the X,Y 
position of the natural object outside the edge of the picture 
referred to the television camera's picture plane and the 

2. A method according to claim 1, characterized in that the 
distance between the object and the basic positions, together 
with the object's x,y,z co-ordinate are determined by trilat- 
eration. 

3. A method according to claim 1, characterized in that 
there are employed 2 fixed basic positions in the preselected 
x,y,z co-ordinate system if the natural object is located for 
every value of t in a plane defined by the fact that one of the 
co-ordinates x,y,z of the natural object is equal to zero, or 
that at least 4 fixed basic positions are employed in the 
preselected x,y,z co-ordinate system if the natural object is 
located for at least one or some values of t in a space defined 
by the fact that none of the co-ordinates x,y,z is equal to 0, 
thus obtaining an unambiguous determination of the natural 
object's x,y,z co-ordinate at time t. 

4. Amethod according to claim 1, characterized in that the 
synthetic object is either created with given attributes for 
size, shape and colour, or that the synthetic object's 
attributes are freely selected, or that the synthetic object's 
attributes are chosen within preselected limits, or that the 
synthetic object's attributes are determined on the basis of 
respective reference values for the attributes. 

5. A method according to claim 4, wherein the synthetic 
object's attributes are determined on the basis of respective 
reference values for the attributes, characterized in that the 
synthetic object's attributes are manipulated automatically 
on the basis of setting values for the television camera such 
as camera angle and zoom setting and on the basis of the 
colour of the background of the natural object as shown in 
the television picture. 

6. Amethod according to claim 1, characterized in that the 
synthetic object is assigned alphanumerical or symbolic 
signs which indicate the value of one or more parameters of 
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the represented natural object, and the parameters may be 
constants which identify the natural object or dynamic 
values such as the object's momentary x,y,z position, dis- 
tance to a freely selected fixed point, course and speed. 

5 7. A method according to claim 1, characterized in that the 
settings of the television camera such as camera angle and 
zoom setting are controlled in approximate real time on the 
basis of the calculated X,Y position of the natural object at 
the time t and referred to the camera's picture plane and the 

to camera vector, with the result that the natural object is 
located within the edge of the picture of the generated 
television picture at any time t, the control being performed 
via a control system assigned to the camera. 

8. Amethod according to claim 7, wherein more than one 
15 camera is employed, characterized in that a choice is made 

via the control system as to which camera should be 
employed to generate the television picture which is shown 

9. A method for generating at least one synthetic track in 
20 a television picture, wherein the synthetic track represents 

the path of a movable, natural object in a natural television 
picture during a given period 8, and wherein a method is 
employed for manipulating the movable, natural object in 
the television picture as indicated in claim 1, characterized 

25 by calculating the path of the natural object on the basis of 
detected positions x,y,z for the a natural object in the 
preselected x,y,z co-ordinate system at the time t, where te6, 
converting the detected positions at the time t to an X,Y 
position in the television camera's picture plane at the time 

30 t, and generating the synthetic track in the television picture 
as the connecting line between all X,Y positions of the 
natural object in the picture planes of the natural television 
pictures which are recorded sequentially during the period 8. 

10. A method according to claim 9, characterized in that 
35 the synthetic track is generated and updated continuously for 

each individual picture which is recorded during the period 
6. 

11. A method according to claim 10, characterized in that 
the synthetic track is generated and displayed as the path of 

40 the natural object during the period 8 in a freely selected 
television picture which forms a background for the natural 
object, the X,Y positions being scaled for each time te8 and 
assigned to the picture plane of the freely selected television 

45 12. A method according to claim 11, characterized in that 
the freely selected television picture is a natural television 
picture, the freely selected television picture forming part of 
the sequence of the natural television pictures which are 
recorded during the period 8, or that the freely selected 

50 television picture is a synthetic television picture. 

13. A method according to claim 9, characterized in that 
the X,Y positions for one or more of the times t are indicated 
on the synthetic track, the indication of the X,Y position in 
the synthetic track being formed as a synthetic object with 

55 given attributes for size, shape and colour, and in such a 
manner that the synthetic object constitutes a representation 
of the natural object recorded by the camera at the time te8. 

14. A method according to claim 13, characterized in that 
the indication of the X,Y positions or the synthetic object is 

60 assigned alphanumerical or symbolic signs which indicate 
the value or one or more parameters for the represented 
natural object, and the parameters may be constants which 
identify the natural object or dynamic values such as the 
object's momentary x,y,z position, distance from a freely 

65 selected fixed point, course and speed at the time te8. 

15. A system for implementing the method for manipu- 
lation of at least one movable, natural object (N) in a natural 
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television picture, wherein the television picture is generated 
by one or more television cameras (K), together with imple- 
mentation of the method for generating a synthetic track in 
a television picture, wherein the synthetic track represents 
the path of a movable, natural object in the television picture 5 
during a given period 8, and wherein a method is employed 
for manipulating the movable, natural object in the televi- 
sion picture, characterized in that the system comprises a 
transponder (T) provided in the natural object (N) and 
arranged to react to an optical, acoustic or electromagnetic to 
signal received by the transponder with transmission of a 
response signal, at least one position module (M) with at 
least 2 position detectors (D) for transmitting optical, acous- 
tic or electromagnetic signals and receiving response signals 
from the transponder (T) and provided in respective basic is 
positions in a preselected x,y,z co-ordinate system, together 
with a signal processor (1) arranged to determine the dis- 
tance between a position detector (D) and the object (N) at 
a time t, a computing module (2) connected with the signal 
process (1) and arranged to calculate the x,y,z co-ordinates 20 
for the object (N) in the preselected co-ordinate system at the 
time I and on the basis of the calculated x,y,z co-ordinates 
for a number of times t to calculate a path for the object (N), 
together with an object vector (V^) given by the distance 
between the camera's (K) lens centre and the object (N) at 25 
the time t, a camera vector (V^-) given by the camera's (K) 
optical axis at the time t and a zoom vector (\Q between the 
camera's lens centre and the point of intersection (Z) 
between the edge of the picture and the plane formed 
between the object vector (V^) and the camera vector (V^ 30 
at the time t, the object vector (V N ) being either located 
between the camera vector (V^) and the zoom vector (V z ) 
or the zoom vector (V z ) between the camera vector (V K ) and 
the object vector (V^,), a camera control system (3) con- 
nected with the computing module (2) and arranged to detect 35 
or generate values for the camera settings, and a manipulator 
module (4) connected with the camera control system (3) 
and the computing module (2) arranged to a) create a 
synthetic object (S) in an X,Y position for the natural object 
(N) in the recorded television picture at the time t, the 40 
synthetic object )S) constituting a representation of the 
natural object (N) recorded by the camera at the time t or b) 
create a symbol n the television picture, the symbol indi- 



cating the X,Y position for the natural object outside the 
edge of the picture at the time t, c) generate and select 
attributes for the synthetic object (S), or d) generate at the 
time t a synthetic track in a recorded or generated television 
picture, the synthetic track representing the path for the 
natural object (N) during a period 6 before or up to the time 
t. 

16. A system according to claim 15, characterized in that 
the system comprises a number of position modules (M) 
each of which is assigned to one or more respective televi- 
sion cameras (K), each position module (M) being arranged 
to cover a predefined area in the x,y,z co-ordinate system. 

17. A system according to claim 15, characterized in that 
the position detector (D) comprises a microwave transceiver 
and an antenna (II) for transmission and receipt of micro- 
wave signals, and that there is further provided a transponder 
T in each of the position detectors (D) and/or in each of the 
cameras (K) respectively. 

18. A system according to claim 17, wherein each said 
position module comprises at least 4 position detectors 
(Dl, . . . D4) for unambiguous determination of the x,y,z 
co-ordinates for the object (N) at the time t. 

19. A system according to claim 15, characterized in that 
the transponder (T) is either a passive transponder, or an 
active transponder, the active transponder being a surface 
wave component (SAW component), and that the response 
signal from the active transponder (T) in each case consti- 
tutes a code which unambiguously identifies the transponder 
by delaying the response signal from the active transponder 
(T) by a predetermined value x d in relation to the time of 
receipt of a signal which causes the response signal to be 
transmitted form the transponder. 

20. A system according to claim 15, characterized in that 
the camera control system (3) comprises a control loop for 
automatic control of camera settings in approximate real 
time, the camera settings being influenced via a regulator 
module (R) provided on the camera (K) which module 
further comprises sensors for detection of the camera 
settings, and/or that the manipulator module (4) comprises a 
device for automatic generation of attributes for the syn- 
thetic object. 
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Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 24-33 and 39-53 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Moengen (US 6,373,508). 

Re claims 24 and 31, Moengen teaches a system (fig. 1) comprising: a communicator (Q 
of fig. 1) configured to receive first data associated with an object (Dl and Kl of fig. 1) and 
second data associated with the object (Camera and GPS, col. 16, lines 5-38), wherein the first 
data is received from a fixed detector (Dl and Kl of fig. 1) configured to detect the first data, 
and wherein the second data is received from a mobile target unit (col. 16, lines 5-36, the natural 
object is equipped with GPS system to determination of the position, and the GPS system and 
radio transmitter and receivers would obviously be considered as a mobile target unit) 
comprising a sensor (the camera detects the second data incorporated with GPS system that has a 
sensor for determination of position of the natural object) configured to detect the second data, 
wherein the mobile target unit (GPS system) is at least one of: mounted in the object, mounted 
on the object, carried in the object, or carried on the object; and a processor configured to 
correlate the first data and the second data to generate object location information (col. 16, lines 
5-36, the animals which are equipped with radio transmitters and GPS receivers, this suggestion 
would obvious to one skill in the art to mount the mobile target unit on the object). 
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Re claim 25, Moengen further teaches wherein the communicator is further configured to 
receive a target unit location (Q of fig. 1), the processor being further configured to determine 
whether the mobile target unit is within a range of the fixed detector (col. 6, lines 10-20). 

Re claims 26 and 51, Moengen further teaches wherein: the object location information 
comprises at least one of object trajectory information, object physical location information, or 
object speed information; and the fixed detector is configured to provide an image of the object 
(fig. 10a and 10b). 

Re claim 27, Moengen further teaches wherein the object is a vehicle (Natural Object 
would obviously be considered as a vehicle). 

Re claims 28 and 33, Moengen further teaches a database configured to maintain a 
plurality of current positions associated with at least one of a plurality of sensors, a plurality of 
mobile target units, or a plurality of objects (P of fig. 1, production unit would obviously has a 
database to contain position information associated with the object). 

Re claim 29, Moengen further teaches wherein the mobile target unit comprises an 
accelerometer configured to provide data indicative of movement of the object to facilitate 
generating the object location information (Transponders, note the active transponder also has to 
be mounted in the natural object, it has to be robust and capable of withstanding jolts and shocks 
as well as relative high accelerations, wherein the transponders are equipped with the natural 
object, col. 16, lines 5-50). 

Re claim 30, Moengen further teaches wherein: the object is an identified good (natural 
object would obviously be an identified good); the mobile target unit comprises a radio- 
frequency identification device (GPS transmitter and receiver, col. 16, lines 5-38); and the fixed 
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detector comprises a camera (Dl and Kl of fig. 1) for observing the identified good, to facilitate 
enabling the sensor and the fixed detector to provide corroborative surveillance of the identified 
good (fig. 1). 

Re claim 39, Moengen further teaches wherein the second data comprises the target unit 
location (col. 16, lines 5-50, GPS and Transponders generate the target unit location). 

Re claim 40, Moengen further teaches wherein the correlating the first data and the 
second data comprises determining compliance with a scheduled object activity (Sport Event, 
Start to Finish, fig. 10a and 10b). 

Re claim 41, Moengen further teaches wherein the correlating the first data and the 
second data comprises determining a movement vector to predict a future location of the object 
(Q of figs 1 and 2, col. 16, lines 5-38, determine the object is out of range). 

Re claim 42, Moengen further teaches a plurality of detectors (Kl-Kn of fig. 1) each 
having a corresponding observation range (camera has observation range), wherein at least one 
of the plurality of detectors is selected to observe the object (Kl of fig. 1, Kl is selected to 
capture an image of the object). 

Re claim 43, Moengen further teaches wherein the first data comprises at least one of an 
image of the object or an identifier associated with the object (Image of the object, fig. 10a and 
10b). 

Re claim 44, Moengen further teaches wherein the first data comprises a plurality of 
images of the object captured at different times (figs. 10a and 10b). 

Re claim 45, Moengen further teaches wherein the second data comprises at least one of 
an image of the object or an identifier associated with the object (figs. 10a and 10b). 
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Re claim 46, Moengen further teaches wherein the second data comprises a plurality of 
images of the object captured at different times (figs. 10a and 10b). 

Re claim 47, Moengen further teaches wherein the object location information is 
determined at least in part based on a fixed detector location (Kl of fig. 1). 

Re claim 48, Moengen further teaches wherein the object location information is 
determined at least in part based on a mobile target unit location (col. 16, lines 5-38). 

Re claims 32 and 49, Moengen further teaches a movement module (1 of fig. 2) 
configured to activate a second fixed detector (K of fig. 2 is activated based on the location of 
the object, N of fig. 2) in response to the object location information. 

Re claims39 5 1 , Moengen further teaches wherein the mobile target unit (col. 5, lines 5- 
30) comprises a locator unit (GPS system) configured to determine the target unit location. 

Re claim 52, Moengen further teaches wherein the fixed detector (K of fig. 2) is 
configured to be selected in response to the processor's correlation of the first data and the 
second data (Dl, D2 of fig. 2, and GPS in col. 16, lines 5-38). 

Re claim 53, Moengen further teaches wherein the fixed detector is further from the 
second fixed detector than from a third fixed detector (Kl, K2, and K3 of fig. 1). 

Response to Arguments 
3 . Applicant's arguments filed 04/02/20 1 0 have been fully considered but they are not 
persuasive. 

The applicant argues that Moengen does not disclose "a processor configured to correlate 
the first data from the fixed detector and the second data from the mobile target unit to generate 
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object location information" as recited in claims 24 and 3 1 ; and Moengen explicitly excludes 
using any image data received through the cameras to determine position. 

The examiner respectfully disagrees with the applicant. It is submitted that Moegen 
teaches a processor (Q and P of fig. 1) configured to correlate (1 and 2 of fig. 2) the first data 
from the fixed detector (the position of the natural object N is detected by the detector and 
camera, Dl and Kl of fig. 1) and the second data from the mobile target unit (col. 16, lines 1 1- 
20, the camera detects the second data incorporated with GPS system that has a sensor for 
determination of position of the natural object) to generate object location information (e.g. figs. 
9a, b, c, note the system for manipulating (4 of fig. 2) the picture of at least one movable, natural 
object in a natural television picture in such a manner that the object's position and movement 
are clearly visible in the television picture, col. 4, lines 51-63; wherein synthetic object represent 
position of the natural object any t time, which indicates the future position of the natural 
object). Note claimed features do not include using any image data received through the 
cameras to determine position. Therefore, the disclosure of Moengen meets the claimed features. 

The applicant argues that claims 25-26, 28-30, 32-33, 39-40, and 42-53 variously depend 
from independent claims 24 and 3 1 . Therefore, Applicant asserts that dependent claims 25-26, 
28-30, 32-33, 39-40, and 42-53 are differentiated from the cited references for at least the same 
reasons stated above for differentiating independent claims 24 and 31, as well as in view of their 
own respective features. Applicant thus requests the Examiner's rejection of claims 25-26, 28-30, 
32-33, 39-40, and 42-53 is withdrawn. The arguments are not persuasive because all limitations 
of claims 25-26, 28-30, 32-33, 39-40, and 42-53 have been addressed above. 
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The applicant argues that Moengen does not disclose or contemplate "wherein the mobile 
target unit is at least one of: mounted in the object, mounted on the object, carried in the object, 
or carried on the object" as recited in claim 31, and as similarly recited in claim 24. 

The examiner respectfully disagrees with the applicant. It is submitted that Moengen 
teaches wherein the second data is received from a mobile target unit (col. 16, lines 5-36, the 
natural object is equipped with GPS system to determination of the position, and the GPS system 
and radio transmitter and receivers would obviously be considered as a mobile target unit) 
comprising a sensor (the camera detects the second data incorporated with GPS system that has a 
sensor for determination of position of the natural object) configured to detect the second data, 
wherein the mobile target unit (GPS system) is at least one of: mounted in the object, mounted 
on the object, carried in the object, or carried on the object (col. 16, lines 5-36, the animals which 
are equipped with radio transmitters and GPS receivers, this suggestion would obvious to one 
skill in the art to mount the mobile target unit on the object). 

The applicant argues that Moengen does not teach the claimed limitations in claim 24. 

The examiner respectfully disagrees with the applicant. It is submitted that Moengen 
teaches wherein the correlating the first data and the second data comprises determining a 
movement vector to predict a future location of the object (Q of figs 1 and 2, col. 16, lines 5-38, 
determine the object is out of range). 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 
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Utke et al. (US 5,346,210) discloses an object locator system. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung Vo whose telephone number is 571-272-7340. The 
examiner can normally be reached on Monday- Wednesday, Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on 571-272-741 8. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
/Tung Vo/ 

Primary Examiner, Art Unit 2621 
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